Imitation Learning in The Game of Go with Joseki Options

Scaling reinforcement learning methods to large, challenging decision making tasks can potentially benefit from integrating domain specific knowledge in a principled manner. This synthesis focuses on applying two forms of domain knowledge about the game of Go to improve learning performance on what continues to be an extremely challenging task. First, learning is bootstrapped by using reinforcement learning to learn to imitate expert Go players. This utilizes databases of expert game records as a source of training experiences. Second, reusable options are automatically created from a joseki database, and reinforcement learning is used to learn when to apply these joseki during a game. Together these improve the performance of a reinforcement learning agent against a much better computer Go player.